Method and apparatus for generating data representative of an image

ABSTRACT

A method and device for generating image data are disclosed herein. One embodiment of the method comprises focusing an image of an object onto a two-dimensional photosensor array, wherein an optical element is located between the object and the two-dimensional photosensor array. The optical element is moved, wherein the moving causes the image focused on the two-dimensional photosensor array to move parallel to the two-dimensional photosensor array. Data representative of the image is generated by the two-dimensional photosensor array while the optical element is moving.

BACKGROUND

Digital cameras and other digital imaging devices generatemachine-readable image data (sometimes referred to simply as “imagedata”) representative of images of objects. The process of generatingimage data representative of an image of an object is sometimes referredto as imaging or capturing an image of the object. During the imagingprocess, a digital camera forms an image of the object onto atwo-dimensional photosensor array. The photosensor array has a pluralityof discrete photodetector elements that are sometimes referred to asphotodetectors. Each of the photodetectors generates an electricalsignal having values proportional to the intensity of light incident onthe photodetectors. The output of the photosensor array is connected toan analog-to-digital converter that is used to convert the electricalsignals generated by the photodetectors into digital numbers. Thedigital numbers output from the analog-to-digital converter areproportional to the intensity of the light incident on thephotodetectors. These digital numbers are sometimes referred to ascounts or raw data. The raw data consists of numbers wherein a highnumber is typically representative of a photodetector that receivedbright light and a low number is typically representative of aphotodetector that received dim light.

In color digital photography, color is typically generated using filtersin a prescribed color-filter array pattern. A filter placed over the topof each photodetector limits the response of the photodetector so thatthe raw data produced is limited to a preselected wavelength of light.These preselected wavelengths of light typically correspond to the threeadditive primary wavelengths or colors of red, green, and blue. The rawdata representative of three colors is processed to generate one pixelin the final image. One common type of color filter uses a Bayerpattern. The Bayer pattern is a four-pixel cluster that consists of apixel that responds to red light, two pixels that respond to greenlight, and a pixel that responds to blue light.

Digital images are generated by sampling a continuous scene or object.The sampling process consists of mapping the scene onto to thetwo-dimensional grid of photodetectors that forms the photosensor array.Due to the discrete nature of the digital imaging process the imagegenerated by a digital camera is subject to certain image anomaliesresulting from the sampling process. One anomaly is aliasing, which isthe generation of false frequencies when an image is undersampled.Aliasing becomes more apparent when an image of an object having highspatial frequency content is imaged.

The highest spatial frequency that may be replicated is one half thesampling frequency, which is referred to as the Nyquist frequency.Frequencies higher than the Nyquist frequency are aliased down to lowerfrequencies. The lower frequency features introduce artifacts into theimage, which can create false images and form moiré patterns in periodicscenes. The anomalies are even greater if the digital camera is used togenerate a video or movie images because the camera typically does notuse all the photodetectors in the array. Thus, the sampling ratedecreases and the effects due to aliasing increase.

SUMMARY

A method and device for generating image data are disclosed herein. Oneembodiment of the method comprises focusing an image of an object onto atwo-dimensional photosensor array, wherein an optical element is locatedbetween the object and the two-dimensional photosensor array. Theoptical element is moved, wherein the moving causes the image focused onthe two-dimensional photosensor array to move parallel to thetwo-dimensional photosensor array. Data representative of the image isgenerated by the two-dimensional photosensor array while the opticalelement is moving.

BRIEF SUMMARY OF THE DRAWINGS

FIG. 1 is a schematic illustration of a side view of an embodiment of animaging system.

FIG. 2 is a schematic illustration of a front view of an embodiment ofthe photosensor array used in the imaging system of FIG. 1.

FIG. 3 is an enlarged view of two of the photodetectors on thephotosensor of FIG. 2.

FIG. 4 is an illustration of a Bayer filter.

FIG. 5 is a front view of an embodiment of the blur filter of FIG. 1.

FIG. 6 is an embodiment of the optical element of FIG. 1 being adithered optically transparent plate.

FIG. 7 is an embodiment of the optical element of FIG. 1 being adithered grating.

FIG. 8 is an embodiment of the optical element of FIG. 1 being atranslated wedge.

FIG. 9 is an embodiment of the optical element of FIG. 1 being atranslated ground-glass plate.

FIG. 10 is an embodiment of the optical element of FIG. 1 being amirror.

FIG. 11 is an embodiment of blur profiles generated in the Y direction.

FIG. 12 is an embodiment of the blur velocity associated with theembodiment of FIG. 11.

FIG. 13 is an embodiment of the blur velocity associated with theembodiment of FIG. 11.

DETAILED DESCRIPTION

A non-limiting embodiment of a block diagram of an imaging system 100 isshown in FIG. 1. The imaging system 100 may be one of many differenttypes of digital imaging devices that generate machine-readable imagedata (referred to herein simply as image data) representative of animage of an object. The generation of image data representative of animage of an object is sometimes referred to as imaging an object orcapturing an image of an object. For example, the imaging system 100 maybe part of a scanner, a digital still camera, or a digital video camera.In the examples provided herein, the imaging system 100 is part of adigital camera that is primarily used to capture still images. Thedigital camera described herein may also have the ability to capturemoving images to create movies or video images.

The imaging system 100 may include a photosensor array 110, a blurfilter 112, a focusing lens 114, a processor 116, and a user interface118. The blur filter 112 may include an optical element 124 and a motiondevice 126, wherein the motion device 126 serves to move the opticalelement 124. It should be noted that other components, not shown,generally associated with imaging systems may be included in the imagingsystem 100. For reference purposes, a coordinate system of x-directions,FIG. 2, y-directions, FIG. 1, and z-directions are used herein. Theimaging system 100 of FIG. 1 is shown imaging or capturing an image ofan object 128.

In summary, the imaging system 100 serves to generate image datarepresentative of an image of an object, such as the object 128. Theimage of the object 128 is focused onto the photosensor array 110, whichgenerates raw data. Raw data as used herein is data generated by thephotosensor array 110. Image data, such as JPEG data, is generated byprocessing the raw data. The raw data consists of a plurality of samplesof the image that are representative of the intensity of light in theimage at sample points. The sample points may be photodetectors locatedon the photosensor array 110.

The generation of image data in a digital imaging device such as theimaging system 100 may be subject to aliasing, which distorts the imageof the object 128 when it is replicated. Aliasing is due, in part, toundersampling of the image of the object. Undersampling, in turn, is dueto the combination of the image of object having high spatialfrequencies and the sampling rate being too low to sample the highspatial frequencies. In order to replicate an image, the samplingfrequency must be at least twice the highest spatial frequency. Thissampling frequency is referred to as the Nyquist frequency. Frequenciesgreater than the Nyquist frequency are aliased to lower frequencies,which introduce artifacts into the replicated image. The artifacts maycreate false images, form moire patterns in periodic scenes, and causeother anomalies in the replicated image.

The imaging system 100 described herein reduces aliasing by use of theblur filter 112. The blur filter 112 blurs the image of the object 128during generation of the raw data, which in turn functions as a low-passfilter and attenuates high frequency components of the image. The blurin the imaging system 100 is achieved by the optical element 124blurring the image, moving the optical element 124 during generation ofthe raw data, or a combination of both. When the optical element 124 ismoved, the image of the object 128 is moved relative to the photosensorarray 110. When the optical element is moved during generation of theraw data the raw data is representative of a blurred image havingreduced spatial frequencies. The amount of movement or blur may beselected depending on the sampling rate used in the generation of thedata. The sampling rate may be selected based on the Nyquist frequencyand other principles. For example, if the imaging system 100 is used togenerate image data representative of still images, a high degree orrate of sampling may be used, which results in little blur beingrequired. As described in greater detail below, a greater amount of blurmay be required in the generation of image data representative of moviesor video images wherein a lower degree or rate of sampling is used.

Having summarily described the imaging system 100, it will now bedescribed in greater detail.

With additional reference to FIG. 2, which is a front view of thephotosensor array 110, the photosensor array 110 may have a plurality ofphotodetectors 130 located on a front surface 132. For illustrationpurposes, the size of the photodetectors 130 have been greatly enlargedin the figures. A photodetector is typically only a few microns wide. InFIG. 1, the photodetectors 130 are illustrated as extending from thesurface 132 of the photosensor array 110. However, it should be notedthat the photodetectors 130 may be flush with the front surface 132 ofthe photosensor array 110 or even recessed therein.

The photodetectors 130 serve to convert light intensities to raw data,which may be electronic signals or numbers. For examples, aphotodetector that receives a high intensity of light may generate rawdata having a high number or value. Likewise, a photodetector thatreceives a low intensity of light may generate image data having a lownumber or value. The raw data may be transmitted to the processor 116 byway of a line 135. The line 135 and all other lines herein may be anydevice that transmits data or signals, such as serial or parallel datalines. The lines may use any medium to transmit data, such as wires,infrared, and radio transmissions.

The photodetectors 130 are arranged into a plurality of rows 134 andcolumns 136. For reference, some of the rows 134 are referenced as thefirst row 138, the second row 140, the third row 142, the fourth row144, and the fifth row 146. FIG. 3 is an enlarged view of two of thephotodetectors 130 of FIG. 2 and serves to show the relationship betweenthe photodetectors 130. As shown in FIG. 3, each of the photodetectors130 may have a diameter D and the photodetectors 130 may be separatedfrom each other by a distance S. The diameter D and the distance S maybe only a few microns. Accordingly, the rows 134 of photodetectors 130may be very close to one another or may abut one another.

Each of the photodetectors 130 may generate raw data based on theintensity of light received by each of the photodetectors 130.Photosensor arrays having photodetectors 130 with large diameters sampleless portions of an image than photosensor arrays having photodetectors130 with small diameters. Likewise, photosensor arrays having largedistances between photodetectors 130 sample less light than photosensorarrays having smaller distances between the photodetectors 130. Asbriefly described above, the blur filter 112 serves to reduce theproblems associated with low sampling rates by spreading or blurring thelight from one portion of the image onto a plurality of photodetectors130. Therefore, the spatial frequencies of the image being captured arereduced.

The photosensor array 110 may have several million photodetectors 130located thereon. Each of the photodetectors 130 generates raw datarepresentative of the intensity of light it receives. When the imagingsystem 100 is used to generate image data representative of stillimages, the imaging system 100 typically has time to process the rawdata generated by all the photodetectors 130. In addition, the imagingsystem 100 typically has enough memory to store the resulting image databy use of electronic data storage devices, not shown. When raw data isbeing processed by all the photodetectors 130, the sampling rate of theimage is relatively high. Therefore, images having relatively highspatial frequencies may be processed and the amount of blur or low-passfiltering required to be applied to the images is relatively low.

Even though the sampling rate is typically high when still images areimaged, some low-pass filtering may be required in order to reducealiasing caused by high spatial frequency components of the images. Forexample, when an imaging system or device generates image datarepresentative of still images, blur of one pixel or photodetector istypically used. One pixel blurring scatters light that would normallyilluminate one photodetector or a group of photodetectors to thesurrounding photodetectors. Thus, the low-pass filtering is achieved byway of slightly blurring the image.

When the imaging system 100 is used to generate image datarepresentative of video images, the imaging system 100 may not have theprocessing speed or memory to process the raw data generated by all thephotodetectors 130. In order to overcome this problem, raw data fromfewer than all the photodetectors 130 is processed, which in effect,reduces the sampling rate and lowers the spatial frequencies that may beprocessed by the imaging system 100. For example, in one embodiment, thephotosensor array 110 generates raw data at a rate of thirty images persecond in order to generate a movie or video. In order to process thislarge amount of raw data, a substantial portion of the raw data may bedisregarded or not generated at all.

In one embodiment of the imaging system 100, raw data from only everythird row of the photodetectors 130 is processed or generated when theimaging system 100 is generating image data representative of videoimages. In an example of this embodiment, raw data generated by thefirst row 138 of photodetectors 130 and the fourth row 144 ofphotodetectors 130 may be processed. Data generated by the second row140 and the third row 142 may be discarded or simply not generated bythe photosensor array 110. Because fewer portions of the image or imagesare sampled, the sampling rate of the image is reduced. The lowersampling rate of the image results in the imaging system 100 havingreduced abilities to process images having high spatial frequencieswithout the replicated images having the above-described anomalies.

When the above-described one pixel of blur is applied to the photosensorarray 110 during the period that the photosensor array 110 generates rawdata, aliasing may still occur due to the significantly lowered samplingrate. For example, if the object 128 is bright and is moving in they-direction, raw data will only be generated during periods that itsimage is focused on every third row of photodetectors 130. If the object128 is relatively large, only portions of the image that are focused onevery third row of photodetectors 130 will be imaged. Upon replicationof the video, the replicated image of the object 128 may appear to flashas it moves in the y-direction. The movement of the object 128 may alsobe improperly replicated as discreet movements rather than a smoothmotion. This discreet movements are due to edges of the object 128 beingimaged only by the rows 134 of photodetectors 130 that generate imagedata. In addition, replicated still images of vertical objects mayappear roped. Roping is caused by the image data sampling along they-direction being inconsistent or not meeting Nyquist criteria. Inaddition, artifacts may be introduced into the image and moire patternsmay appear.

It should be noted that other imaging techniques may sample fewer thanall the photodetectors 130. For example, raw data from various columns136 of photodetectors 130 may not be generated or processed. In anotherexample, raw data from various photodetectors 130 may not be generatedor processed. These other imaging techniques will produce replicatedimages that may have anomalies similar to those described above. Asdescribed in greater detail below, the moving blur filter 112 describedherein serves to overcome or reduce many of the above-describedanomalies.

Having summarily described the photosensor array 110, other componentsof the imaging system 100 will now be described in greater detail. Thephotosensor array 110 may have a color filter, not explicitly shown,located adjacent the photodetectors 130. The color filter serves tofilter light so that only light of preselected frequencies intersectsspecific photodetectors 130. The photodetectors 130 then generate rawdata proportional to the intensity of the specific frequency componentsof light that passes through the color filter. The raw data representingdifferent colors is combined and/or processed during replication of theimage to reconstruct a color image. It should be noted that variousdemosaicing algorithms may be applied to the raw and image data in orderto replicate the image of the object 128.

One embodiment of a color filter uses a Bayer pattern as shown in FIG.4. The Bayer pattern can be considered to arrange the photodetectors130, FIG. 3, into groups or tiles of four photodetectors 130, whereinthere are two green photodetectors (designated by the letter ‘G’) forevery one red photodetector (designated by the letter ‘R’) and one bluephotodetector (designated by the letter ‘B’). The raw data generated bythe groups of four photodetectors is combined during processing tocreate a single picture element or pixel of the replicated image. Theuse of a color filter further decreases the sampling rate of thephotosensor array 110 by reducing the number of photodetectors thatgenerate image data representative of the object. More specifically, thecolor filter causes the image to be divided into specific spectralcomponents wherein specific photodetectors generate image data ofpreselected spectral components. Thus, the number of photodetectors thatgenerate image data representative of the whole image is effectivelyreduced.

Referring again to FIG. 1, the focusing lens 114 serves to focus animage of the object 128 onto the photodetectors 130. The focusing lens114 is shown in FIG. 1 as a being a single optical element. The focusinglens 114, however, may be a plurality of lenses commonly used in digitalimaging devices. In one embodiment of the imaging system 100, thefocusing lens 114 moves in the z-direction in order to enlarge ordecrease the size of the image focused onto the photosensor array 110.Thus, the focusing lens may provide for a zoom feature of the imagingsystem 100 in addition to focusing a sharp image of the object 128 ontothe photodetectors 130.

The processor 116 serves to process raw data generated by thephotosensor array 110. The processor 116 may transform the raw datagenerated by the photosensor array 110 into formats commonly used byreplication devices such as video monitors and printers. The processor116 may also analyze the raw data to determine the spatial frequency ofthe image of the object 128 as represented by the raw data or theprocessed image data. As described in greater detail below, the spatialfrequency of the raw data is a factor in determining the amount of blurthe blur filter 112 creates by moving the optical element 124. Withrespect to blur, the processor 116 may also control the movement of theoptical element 124 via the motion device 126, which is also describedin greater detail below.

The user interface 118 may be connected to the processor 116 by a line150. The user interface 118 serves to enable the user of the imagingsystem 100 to input data concerning the image for which image data isbeing generated. For example, the user may input information regarding adesired sampling rate or blur to be applied during generation of theimage data. There are many different methods and devices for providingthe user interface 118. For example, the user interface 118 may haveswitches that may be activated by the user. In another example, theimaging system 100 may have a touch screen, not shown, wherein the userinterface 118 is associated with the touch screen.

Having described some of the components of the imaging system 100, theblur filter 112 will now be described in greater detail. In summary, theblur filter 112 serves to blur the image of the object 128 so as toattenuate high spatial frequency components of the image of the object128, which in turn reduces some anomalies associated with digitalimaging. A front view of an embodiment of the blur filter 112 is shownin FIG. 5 and a side view of the blur filter 112 is shown in FIG. 1. Theblur filter 112 includes the optical element 124, which is connected tothe motion device 126 by way of a connector 154. The connector 154 maybe part of the motion device 126 and may, as an example, be adhered tothe optical element 124. The motion device 126 is connected to theprocessor 116 by way of a line 158. As briefly described above, theprocessor 116 may control the motion generated by the motion device 126.

The motion device 126 is a device that moves the optical element 124. Inthe embodiment of the imaging system 100, FIG. 1, described above, themotion device 126 moves the optical element 124 in the y-direction.Other embodiments of the motion device 126 described below may move theoptical element 124 in the x-direction or the y-direction. Someembodiments of the motion device 126 may also rotate or pivot theoptical element 124. Some other embodiments of the motion device 126 mayalso move the optical element in a combination of directions. The motiondevice 126 may use, as examples, piezoelectric devices orelectromechanical devices, to move the optical element 124.

Examples of the optical element 124 are shown in FIGS. 6-10. The opticalelement of FIG. 6 is a dithered, optically-transparent plate 170. Withadditional reference to FIG. 1, the motion device 126 may cause thedithered, optically-transparent plate 170 to move along the z axis. Thismovement causes an image of the object 128 focused onto the photosensorarray 110 to be blurred along both the x and y axes, FIG. 2.

FIG. 7, shows an example of the optical element being a dithered grating174. The dithered grating 174 may have a dithered portion 176 that mayface the photosensor array 110 of FIG. 1. In one embodiment, the motiondevice 126, FIG. 1, may cause the dithered grating 174 to move along they axis, which will cause an image of the object 128 to be blurred alongthe y axis relative to the photosensor array 110.

FIG. 8, shows an example of the optical element being a translated wedge180. The translated wedge 180 may have a first surface 182 and a secondsurface 184 that are not parallel to each other. Either the firstsurface 182 or the second surface 184 may face the photosensor array 110of FIG. 1. In one embodiment, the motion device 126 may cause thetranslated wedge 180 to move along the y axis, which will cause an imageof the object 128 to be blurred along the y axis relative to thephotosensor array 110.

FIG. 9 shows a translated, ground-glass plate 188 that may be movedalong the y axis to blur an image. FIG. 10 shows a mirror 190 used toblur an image. A light path 192 may reflect from the mirror 190 as themirror 190 pivots about a pivot point 194. The pivoting causes the lightpath 192 to move so as to blur the image. The pivoting may enable themirror 190 to cause blurring in a plurality of different directions. Forexample, the mirror may pivot about a two axes (not shown).

Referring again to FIG. 1 and as described herein, the blur of the imageof the object 128 is achieved by moving the optical element 124.However, additional blur may be achieved by using an optical element 124that blurs without being moved. For example, the material of the opticalelement 124 may be a birefrigant quartz element or a phase-noise blurfilter. In these examples of the optical element 124, some of the bluris preselected depending on the material used in the construction of theoptical element 124. Additional blur may be provided for the imagingsystem 100 by moving the optical element 124 during generation of rawdata as described herein.

Having described the structure of the imaging system 100, the operationof the imaging system 100 will now be described.

The embodiment of imaging system 100 described herein has two modes. Afirst mode, sometimes referred to as a still mode, generates stillimages and a second mode, sometimes referred to as a video mode,generates video images. During generation of still images, the imagingsystem 100 may use raw data generated by all the photodetectors 130. Insuch a situation, the sampling rate is high. Therefore, very little bluris required to attenuate the effects of aliasing in images containinghigh frequency components. In some embodiments described below,differing amounts of blur are used depending on the spatial frequencycontent of the image.

Movies or video images, on the other hand, typically do not use raw datagenerated by all the photodetectors 130. Use of all the photodetectors130 is very time consuming and typically uses more memory than ispractical. For example, generation of raw data for video imagestypically requires that the photosensor array 110 generate raw datarepresentative of thirty images or frames every second. If all thephotodetectors 130 were used in this process, most practical portablememory devices used in still cameras would be quickly overwhelmed.

With additional reference to FIG. 4, raw data representative of videoimages is typically generated by using less than all the rows 134 ofphotodetectors 130. For example, every third row 134 of photodetectors130 may generate raw data that is to be processed. Use of every thirdrow uses only one third of the photodetectors 130 on the photosensorarray 110, but maintains the Bayer pattern with regard to the colorfilter. If raw data is generated by the other rows 134, it may bedeleted or disregarded during processing. In either event, the raw datagenerated by these rows may not be stored and, therefore, does notoccupy memory or increase the processing time.

As described above, the use of less than all the photodetectors 130degrades the quality of the replicated image by reducing the samplingrate when the raw data is generated. In order to reduce the imagedegradation, the image is blurred on the photosensor array 110. Theamount of blurring, direction in which the blur occurs, and the speed ofthe blur is sometimes cumulatively referred to as the blur profile.Several blur profiles in addition to the ones described herein may beapplied to the image.

Some examples of blur profiles are illustrated in FIG. 11. Forillustration purposes, the blur profiles of FIG. 11 are shown relativeto a column of photodetectors 200 extending in the Y direction. Thecolumn of photodetectors 200 represents a column 136, FIG. 4, of theabove-described Bayer pattern. As described above, when the imagingsystem 100, FIG. 1 is in a video mode, the photosensor array 110 mayonly process every third row of photodetectors 130. The photodetectors130 in the column of photodetectors 200 that may be processed in thevideo mode are circled and are referred to as the active photodetectors.The active photodetectors are referenced as a green photodetector 210, ared photodetector 212, a green photodetector 214, a blue photodetector216, a green photodetector 218, a red photodetector 220, a greenphotodetector 222, a blue photodetector 224, and a green photodetector226.

One of the embodiments shown in FIG. 11 is a no motion embodiment 230where no motion blur is applied to the image. In the no motionembodiment 230, the optical element 124, FIG. 1, remains stationaryrelative to the photosensor array 110 during generation of the raw data.In the embodiment described herein, the optical element 124 is a blurfilter. Therefore, some blurring of the image does occur, however asdescribed herein, the blurring may not be enough to improve the qualityof video images and is used primarily in generating raw datarepresentative of still images. The plurality of lines 232 arerepresentative of the areas of the photosensor array 110 that areilluminated for a particular blur profile during generation of the rawdata. In addition, the lines 232 indicate the photodetectors 130 thatreceive light for the different blur profiles. For example, lightdirected to the red photodetector 212 is blurred so as to also intersectits surrounding green photodetectors. These green photodetectors,however, do not generate raw data that is processed when the imagingsystem 100, FIG. 1, is in the video mode. Likewise, light directed tothe green photodetector 214 is blurred to also intersect its surroundingblue and red photodetectors, which do not generate raw data when theimaging system 100, FIG. 1, is in the video mode.

With the limited blurring provided by the no motion embodiment 230,replicated video images may be distorted. For example, if an objecthaving a high blue color content moves in the Y direction, it will onlybe imaged by approximately every tenth photodetector. More specifically,as the blue object (or an edge of the blue object) moves in the Ydirection, it will be imaged by the blue photodetector 216 when it is insuch a position. As it continues past the blue photodetector 216, itwill not be imaged again until it is in a position to be imaged by theblue photodetector 224. The blue object will appear to jump from oneposition to another when the replicated video is played.

Another problem occurs with a moving object (or an edge of an object)that has a combination of many color components. When the object islocated so that the blue photodetector 216 may image the object, onlythe blue component of the object will be imaged. As the object movestoward a position where the green photodetector 218 images the object,only the green component is imaged. As this process continues, the imagedata generated by the image of the object, or the edge of the object,will change color as the object moves. The replicated video will displayan object that changes color as it moves.

As described above, the optical element 124, FIG. 1, of the imagingsystem 100 moves as raw data is being generated in order to reduce theaforementioned problems. One embodiment of the movement of the opticalelement 124 is illustrated as the first motion embodiment 236. In thefirst motion embodiment 236, the optical element 124 causes lightreflected from a single point to move between three activephotodetectors 130 during the period that raw data is being generated.The amount in which light moves between the photodetectors 130 is one ofthe factors that determines the blur profile. The blur profile may alsoinclude the speed at which the movement occurs.

A plurality of lines 238 illustrate the blurring in the Y directioncaused by the optical element 124. For example, a first line 240illustrates the blurring associated with light that would otherwise besolely incident to the red photodetector 212. The light that would beincident to a specific photodetector is indicated by an x. As shown, thefirst motion embodiment 236 cause the light is blurred between the greenphotodetector 210 and the green photodetector 214. Therefore, light thatwould otherwise only have its red component imaged also has its greencomponent imaged. As described in greater detail below, the amount ofgreen to red imaging may be selected by other factors in the blurprofile. For example, the speed of the motion blur and the time thelight spends on each active photodetector affects the blur profile.

A second line 242 illustrates the blurring associated with light thatwould otherwise be solely incident to the green photodetector 214. Asshown, the first motion embodiment 236 causes light that would otherwisebe solely incident to the green photodetector 214 to be incident alsowith the red photodetector 212 and the blue photodetector 216.Therefore, light that would otherwise only have its green componentsimaged also has its blue and red components imaged.

A third line 244 illustrates the blurring associated with light thatwould otherwise be solely incident to the blue photodetector 216. Asshown, the first motion embodiment 236 causes light that would otherwisebe solely incident to the blue photodetector 216 to be incident alsowith the green photodetector 214 and the green photodetector 218.Therefore, light that would otherwise only have its blue componentsimaged also has its green and red components imaged.

The first motion embodiment 236 provides for continual imaging of anobject by enabling all portions of the image of the object to becontinually incident with the photodetectors 130 even as the objectmoves. Thus, the replicated image of the object will be less likely todisappear and later reappear than it would be without the motion blur.In addition, the color components of the image of the object are morecontinually imaged than they would without the motion blur. Therefore,the replicated image of the object will be less likely to change coloras it moves than it would be without the motion blur. It should be notedthat with the first motion embodiment 236, the blue and red colorcomponents of the replicated image may vary slightly because they arenot necessarily continually imaged. The green component, however, willbe more constant.

It should be noted that the Bayer color filter has twice as many greenphotodetectors as red and blue photodetectors. The first motionembodiment 236, however, may cause the green photodetectors to receivean even greater proportion of light than without the motion. In order toremedy the increase in light received by the green photodetectors, thedata values associated with the green photodetectors may be decreasedduring image processing. For example, the raw data generated by thegreen photodetectors may be scaled down in order to compensate forhigher values associated with the increased proportion of light theyreceive.

A second motion embodiment 250 is also shown in FIG. 11. The secondmotion embodiment 250 is similar to the first motion embodiment 236,however, the blur profile causes a higher degree of motion than thefirst motion embodiment 236. The second motion embodiment 250 isillustrated by a plurality of lines 252. The plurality of lines 252include a first line 254, a second line 256, and a third line 258. Aswith the lines 238, the lines 252 represent the amount and direction ofblur provided by the second motion embodiment 250.

With regard to the first line 254, light that would otherwise only beimaged by the green photodetector 214 is also imaged by the greenphotodetector 210, the red photodetector 212, the blue photodetector216, and the green photodetector 218. Therefore, light that wouldnormally only be imaged by the green photodetector 214 is also imaged byred and blue photodetectors. Thus, the full spectrum of light incidentto the green photodetector 214 may be imaged. With regard to the secondline 256, light that would otherwise only be imaged by the bluephotodetector 216 is also imaged by the red photodetector 212, the greenphotodetector 214, the green photodetector 218, and the redphotodetector 220. Again, light that would otherwise only be imaged bythe blue photodetector 216 has its full spectrum imaged. With regard tothe third line 258, light that would otherwise be image only by thegreen photodetector 218 is also imaged by the green photodetector 214,the blue photodetector 216, the red photodetector 220, and the greenphotodetector 222. As with the aforementioned examples, the light thatwould otherwise only have its green color component imaged has all itscolor components imaged.

The second motion embodiment 250 provides for more inclusive spectralimaging of an object by having light imaged by more of thephotodetectors 130. More specifically, light incident to any of theactive photodetectors 130 will have all its color components imaged. Aswith the first motion embodiment 236 the pixel values generated by thephotodetectors 130 may have to be scaled to compensate for varyingintensities of light that intersect red, green, and blue photodetectorscaused by the second motion embodiment 250. While the proportion of red,green, and blue photodetectors imaging an object may remain the same aswith the Bayer pattern, the speed at which the blur occurs and otherblur profile characteristics may require scaling of the raw data values.

In practice, the second motion embodiment 250 improves the imaging ofthe color components of an object. This results in the colors of theobject being more accurately imaged and maintained even as the objectmoves. Therefore, color transitions of an image of an object will remainrelatively constant, even as the object moves. In addition, the colorsof the image of the object not in the vicinity of color transitions willremain constant because of the imaging of all the color components ofthe object. The second motion embodiment 250, however, may cause theimage of the object to be less detailed than other imaging techniques.In an embodiment described below, the amount of blur is selected basedon the image of the object, so as to provide the sharpest possible imagewhile maintaining color consistency.

FIG. 11 also shows a third motion embodiment 260 that blurs less thanthe first motion embodiment 236 and the second motion embodiment 250. Aplurality of short lines 262 represent the blur of the third motionembodiment 260. The lines 262 include a first line 264, a second line266, and a third line 268.

With regard to the first line 264, light that would otherwise beincident to the green photodetector 210, the red photodetector 212, orphotodetectors 130 located therebetween, is imaged by both the greenphotodetector 210 and the red photodetector 212. With regard to thesecond line 266, light that would otherwise be incident to the redphotodetector 212, the green photodetector 214, or photodetectors 130located therebetween, is imaged by both the red photodetector 212 andthe green photodetector 214. With regard to the third line 268, lightthat would otherwise be incident to the green photodetector 214, theblue photodetector 216, or a photodetectors 130 located therebetween, isimaged by both the green photodetector 214 and the blue photodetector216.

The third motion embodiment 260 provides minimal blurring, but even thisminimal blurring may be enough to greatly improve some images. Forexample, an image having relatively low spatial frequencies may onlyneed slight blurring in order to reduce the effects associated withaliasing. In such a situation, the third motion embodiment 260 may beused.

Having described some embodiments of the amount of movement of theoptical element 124, FIG. 1, some embodiment of the velocities of theoptical element 124 will now be described.

In one embodiment of the imaging system 100, FIG. 1, the optical element124 moves incrementally. Thus, beams of light passing through theoptical element 124 are blurred incrementally relative to thephotosensor array 110. The incremental movement may slow down when beamsof light are incident with active photodetectors 130 and may speed upwhen the beams of light are incident with inactive photodetectors 130.This incremental blurring be achieved by changing the speed of theoptical element 124 during its movement cycle as described in greaterdetail below.

A graph showing an embodiment of an incremental velocity of the opticalelement 124 of FIG. 1 used in conjunction with the first motionembodiment 236, FIG. 11, is shown in FIG. 12. The graph illustrates thesecond line 242 of FIG. 11, which blurs a beam of light 270 that wouldotherwise be incident solely with the green photodetector 214. The beamof light 270 is blurred so that it is incident with photodetectors 130extending between the red photodetector 212 and the blue photodetector216 as shown in FIG. 12. Thus, the graph of FIG. 12 shows the velocityof the movement of the beam of light 270 as it moves between the redphotodetector 212 and the blue photodetector 216.

As the beam of light moves back and forth between the red photodetector212 and the blue photodetector 216 it must stop to change directions.The change in directions causes the beam of light to slow down relativeto the photosensor array 110, FIG. 1, which is indicated by the slopes274 and 276 in the graph. When the beam of light slows down, it becomesincident on a photodetector for a longer period, which results in thephotodetector imaging the beam of light for a longer period. Withrespect to the graph of FIG. 12, the red photodetector 212 and the bluephotodetector 216 may image more of the beam of light 270 than the greenphotodetector 214 because of the time spent by the green photodetector214 slowing down and changing direction. The values of the raw datagenerated by the photodetectors 130 may be scaled to compensate for theabove-described different imaging times.

Another example of moving the beam of light 270 is shown by the graph ofFIG. 13. The embodiment of FIG. 13 uses faster acceleration anddeceleration of the movement of the beam of light 270 than theembodiment of FIG. 12. In addition, the movement of the beam of light270 is slowed when it is in the vicinity of the green photodetector 214.This embodiment enables the beam of light 270 to image the activephotodetectors 130 for the same amount of time. Therefore, the raw dataproduced by the photodetectors 130 may require less scaling or morepredictable scaling.

Having described some embodiments of the amount and velocity of theblur, which are collectively referred to as the blur profile, selectionof the blur profile will now be described. Referring to FIG. 1, in oneembodiment of the imaging system 100, the processor 116 analyzes rawdata produced by the photodetectors 130 and selects the blur profileaccordingly. In another embodiment of the imaging system 100, a userselects the blur profile by way of the user interface 118.

Selection of the blur profile may depend on whether the imaging system100 is in a mode for generating image data representative of stillimages or video images. As described above, the imaging system 100typically does not use all the photodetectors 130 during the generationof image data representative of video images. Therefore, the blur may beincreased in order to compensate for the reduction in the sampling ofthe images. In one embodiment of the imaging system 100, the blur isautomatically increased when the imaging system 100 is used to generateimage data representative of video images. Likewise, the blur may beautomatically reduced when the imaging system 100 is used to generateimage data representative of still images.

In another embodiment of the imaging system 100, the amount of blur isdetermined by analyzing the image of the object 128 prior to capturingthe image. For example, the spatial frequencies of the image may beanalyzed. When an image is determined to have high spatial frequencies,the amount of blur may be increased or set to a preselected high valueprior to capturing the image. Likewise, when an image is determined tohave low spatial frequencies, the amount of blur may be decreased or setto a preselected low value prior to capturing the image.

In another embodiment of the imaging system 100, a user of the imagingsystem 100 may adjust the amount of blur. In one such embodiment, theuser may adjust the blur depending on the object being imaged. Forexample, a user may decide to increase or decrease blur depending onexperience and personal preference. In another such embodiment, theimaging system 100 may display the image that is to be captured. Theuser may then adjust the blur and see the effects on the image as theblur is being adjusted. Therefore, the user may select a desired blurprofile after reviewing several blur profiles.

1. A method of generating data representative of an image, said methodcomprising: focusing an image of an object onto a two-dimensionalphotosensor array using a lens, wherein an optical element is locatedbetween said object and said two-dimensional photosensor array; movingsaid optical element, wherein said moving causes said image focused onsaid two-dimensional photosensor array to move; and generating datarepresentative of said image using said two-dimensional photosensorarray while said optical element is moving.
 2. The method of claim 1,wherein said moving comprises moving said optical element if saidgenerating data comprises generating data representative of movingimages.
 3. The method of claim 1, wherein said moving comprises movingsaid optical element a first amount if said generating data comprisesgenerating data representative of a still image, and wherein said movingcomprises moving said optical element a second amount if said generatingdata comprises generating data representative of video images, whereinsaid first amount is less than said second amount.
 4. The method ofclaim 1, wherein said two-dimensional photosensor array has a firstnumber of photodetectors located thereon and wherein said generatingdata comprises generating data using a second number of photodetectorsof said two-dimensional photosensor array while said optical element ismoving, said second number being less than said first number.
 5. Themethod of claim 4, wherein said moving comprises moving said opticalelement an amount so that a portion of said image intersects at least afirst and at least a second of the photodetectors that generate dataduring the period that said two-dimensional photosensor array generatesdata.
 6. The method of claim 1, wherein said two-dimensional photosensorarray comprises a plurality of rows of photodetectors, wherein a firstrow of photodetectors and a second row of photodetectors generate data,wherein at least one row of photodetectors is located between said firstrow and said second row, and wherein said moving comprises moving saidoptical element so that at least a portion of said image is focused onsaid first row of photodetectors and said second row of photodetectorswhile said two-dimensional photosensor array generates data.
 7. Themethod of claim 6, wherein said two-dimensional photosensor arrayfurther comprises a third row of photodetectors that generate imagedata, and wherein said moving comprises moving said optical element sothat at least a portion of said image is focused on said first row ofphotodetectors, said second row of photodetectors, and said third row ofphotodetectors while said data is being generated.
 8. The method ofclaim 1, wherein said moving further comprises moving said opticalelement at a first velocity when a portion of said image is focused on afirst photodetector and moving said optical element at a second velocitywhen said portion of said image is focused on a second photodetector. 9.The method of claim 8, wherein said first photodetector generates dataand said second photodetector does not generate data and wherein saidfirst velocity is slower than said second velocity.
 10. The method ofclaim 1, wherein said optical element comprises a dithered opticallytransparent plate.
 11. The method of claim 10, wherein said movingcomprises moving said dithered optically transparent plate in adirection substantially normal to said two-dimensional photosensorarray.
 12. The method of claim 1, wherein said optical element comprisesa dithered grating.
 13. The method of claim 12, wherein said movingcomprises moving said dithered grating in a direction substantiallyparallel to said two-dimensional photosensor array.
 14. The method ofclaim 1, wherein said optical element comprises a translated wedge. 15.The method of claim 14, wherein said moving comprises moving saidtranslated wedge in a direction substantially parallel to saidtwo-dimensional photosensor array.
 16. The method of claim 1, whereinsaid optical element comprises a translated ground-glass plate.
 17. Themethod of claim 16, wherein said moving comprises moving said translatedground-glass plate in a direction substantially normal to saidtwo-dimensional photosensor array.
 18. The method of claim 1, whereinsaid optical element comprises a mirror.
 19. The method of claim 18,wherein said moving comprises pivoting said mirror relative to saidtwo-dimensional photosensor array.
 20. An imaging device comprising: atwo-dimensional photosensor array, wherein data representative of animage focused onto said two-dimensional photosensor array is generatableby said two-dimensional photosensor array; a light path extending from apoint not proximate said two-dimensional photosensor array to saidtwo-dimensional photosensor array; a lens located in said light path;and an optical element located in said light path, said optical elementbeing movable relative to said two-dimensional photosensor array; saidmovable optical element being movable during a period when saidtwo-dimensional photosensor array generates data; wherein movement ofsaid optical element causes said light path to move relative to saidtwo-dimensional photosensor array.
 21. The imaging device of claim 20:wherein said two-dimensional photosensor array comprises of a firstnumber of photodetectors; wherein said imaging device has a firstoperative mode when data is generatable by said first number ofphotodetectors; wherein said imaging device has a second operative modewhen image data is generatable by a second number of photodetectors; andwherein said first number is greater than said second number.
 22. Theimaging device of claim 20: wherein said two-dimensional photosensorarray comprises a first row of photodetectors, a second row ofphotodetectors, and at least one row of photodetectors located betweensaid first row of photodetectors and said second row of photodetectors;wherein said first row of photodetectors and said second row ofphotodetectors generate image data; and wherein said optical element ismovable so that a portion of an image is focusable on said first row ofphotodetectors and said second row of 10 photodetectors when saidtwo-dimensional array photosensor generates data.
 23. The imaging deviceof claim 20 and further comprising a piezoelectric device operativelyconnected to said optical element.
 24. The imaging device of claim 20,wherein said optical element is pivotable relative to saidtwo-dimensional photosensor array.
 25. The imaging device of claim 20,wherein said optical element comprises a dithered optically transparentplate.
 26. The imaging device of claim 25, wherein said ditheredoptically transparent plate is movable in a direction substantiallynormal to said two-dimensional photosensor array.
 27. The imaging deviceof claim 20, wherein said optical element comprises a dithered grating.28. The imaging device of claim 27, wherein said dithered grating ismovable in a direction substantially parallel to said two-dimensionalphotosensor array.
 29. The imaging device of claim 20, wherein saidoptical element comprises a translated wedge.
 30. The imaging device ofclaim 29, wherein said translated wedge is movable in a directionsubstantially parallel to said two-dimensional photosensor array. 31.The imaging device of claim 20, wherein said optical element comprises atranslated ground-glass plate.
 32. The imaging device of claim 31,wherein said translated ground-glass plate is movable in a directionsubstantially normal to said two-dimensional photosensor array.
 33. Theimaging device of claim 20, wherein said optical element comprises amirror.
 34. The imaging device of claim 33, wherein said moving saidmirror is pivotable relative to said two-dimensional photosensor array.35. An imaging device comprising: a photosensing means for generatingdata representative of an image focused onto said photosensing means; afocusing means for focusing said image onto said photosensing means; anda movement means for moving said image focused onto said photosensingmeans when said photosensing means is generating data.
 36. The imagingdevice of claim 35 wherein said movement means comprises a blurringmeans connected to a movement device and wherein said blurring means islocated proximate said photosensing means.